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Hitting Time Distribution for Skip-Free Markov 
Chains: A Simple Proof 

Wenming Hon^ Ke Zhoiil 

m ■ 

^ ! Abstract 

A well-known theorem for an irreducible skip-free chain with absorbing 

■ state d, under some conditions, is that the hitting (absorbing) time of 
. state d starting from state is distributed as the sum of d independent 

geometric (or exponential) random variables. The purpose of this paper 

■ is to present a direct and simple proof of the theorem in the cases of 
both discrete and continuous time skip- free Markov chains. Our proof 

[ is to calculate directly the generation functions (or Laplace transforms) 

' of hitting times in terms of the iteration method. 
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■ 1 Introduction 
^ ! 

' The skip-free Markov chain on is a process for which upward jumps may be only of unit 

size, and there is no restriction on downward jumps. If a chain start at 0, and we suppose d is 
an absorbing state. An interesting property for the chain is that the hitting time of state d is 
distributed as a sum of d independent geometric (or exponential) random variables. 
^ ■ There are many authors give out different proofs to the results. For the birth and death 

, chain, the well-known results can be traced back to Karlin and McGregor [^, Keilson [8], [9]. 

Kent and Longford [10] proved the result for the discrete time version (nearest random walk) 
although they have not specified the result as usual form (section 2, [10]). Fill [4] gave the first 
stochastic proof to both nearest random walk and birth and death chain cases via duality which 
was established in [2] . Diaconis and Miclo [3] presented another probabilistic proof for birth and 
death chain. Very recently. Gong, Mao and Zhang [6] gave a similar result in the case that the 
state space is Z"*", they use the well established result to determine all the eigenvalues or the 
spectrum of the generator. 
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For the skip-free chain, Brown and Shao [T] first proved the result in continuous time situa- 
tion. By using the duahty, Fill [5] gave a stochastic proof to both discrete and continuous time 
cases. The purpose of this paper is to present a direct and simple proof of the theorem in the 
cases of both discrete and continuous time skip- free Markov chains. Our proof is to calculate di- 
rectly the generation functions (or Laplace transforms) of hitting times in terms of the iteration 
method. 

Theorem 1.1. For the discrete-time skip-free random walk: 

Consider an irreducible skip-free random walk with transition probability P on {0, 1, • • • ,d} 
started at 0, suppose d is an absorbing state. Then the hitting time of state d has the gen- 
eration function 

"Pdis) = n 

i=0 

where Aq, • • • , A^-i are the d non-unit eigenvalues of P. 

In particular, if all of the eigenvalues are real and nonnegative, then the hitting time is dis- 
tributed as the sum of d independent geometric random variables with parameters 1 — Aj . 



(1 - Xi)s 
1 - AjS 



Theorem 1.2. For the skip-free birth and death chain: 

Consider an irreducible skip-free birth and death chain with generator Q on {0, 1, • • • ,d} started 
at 0, suppose d is an absorbing state. Then the hitting time of state d has the Laplace transform 



d-l , 
Ai 



^d{s) = n Y 



A,:s' 



where Aj are the d non-zero eigenvalues of —Q. 

In particular, if all of the eigenvalues are real and nonnegative, then the hitting time is 
distributed as the sum of d independent exponential random variables with parameters Aj. 



2 Proof of Theorem 1.1 

Define the transition probability matrix P as 



91,0 



Po 
ri 



Pi 



-1,0 Qd~l,l Qd~l,2 



V 



rd~i Pd~i 

1 / 



{d+l)x(d+l) 



and for < n < d — 1, P„ denote the first n + 1 rows and first n + 1 lines of P. 

Let Tj^j+j be the hitting time of state i + j starting from i. By the Markov property, we have 



Ti,i+j — Ti,i+1 + Ti+l,i+2 + ' ' ' + Ti+j-l^i+j. 

If is the generation function of Tj^j+i, 

fi,i+i{s) = Es^'-»+i for < i < d - 1, 



(2.1) 
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(gH) says that 

fi,i+j{s) = fi,i+i{s) • /i+i_i+2(s) fi+j-i^i+j{s), for 1 < j < d - i. 

Let 

50,0(5) = 1, gi,i+j[s) = -— s^, for 1 < J < d - z. 

Ji,i+j\S) 

Lemma 2.1. Define In as a {n + 1) x (n + 1) identity matrix. We have 

go,n+i{s) = det{In - sPn), for < n < d - I. (2.2) 

Proof. We will give a key recurrence to proof this lemma. By decomposing the first step ,the 
generation function of r„^„+i satisfy 

fn,n+l{s) = rnSfn,n+l{s) + PnS + Qn^n-lS fn-l,n+l{s) + qn,n-2S fn-2,n+l{s)+ 
\- qn,OSfo,n+lis)- 

Recall the definition of gi^i^j{s), substitute it into the formula above, we have 

fi'0,n+l('5) = (1 - rnS)go,n{s) - gn,n~lS^Pn-l50,n-l (s) " qn,n-2S^Pn-2Pn~igO,n~2{s)- , 

n+1 , X (2-4) 
9n,0S POPl • • •Pn-l50,0(s)- 



Use the notation Ai^j to denote the algebraic complement of the position (i + 1, j' + l) in I„ — sP„. 
By expanding the bottom row of the matrix, we obtain 

det(/„ - SP„) = (1 - rnS)An^n - qn,n-\sAn^n-\ " gn,n-2sA,n-2 9^,05^^,0- (2-5) 

By some calculation, we can deduce 

An^n = det(/„_i - sPn-l), Anfl = PoPl ■ • -pn-is", 

and for 1 < i < n, 

■^n,n—i — Pn~iPn~i+l ' ' ' Pn—l^ det(/n_j_i sPn~i~\) • 

Now we prove the lemma by induction. At first, go,i(s) = det(/o — sPq) = 1 — r^s. If (j2.2p holds 
for n < k, we calculate 170,^+1 (s). By (j2.4p . 

5o,fe+i(s) = (1 - rfcs)det(Ifc_i - sPk-i) - qk,k~is'^Pk~idet{Ik-2 - sPk-2)- 

qk,k-2S^Pk-2Pk~idet{Ik-3 - sPk^s) qk,os''^^PoPi ■ ■ ■ Pk^i- 

Formula (|2.5p tell us that gQ^k+i[s) = det(/fc — sPk)- The proof is complete. □ 

Proof of Theorem 1.1 Denote ^d{s) the generation function of ro^d, then fdis) = fo,d{s). By 
(I2ID and (H^I) , we have 

^d{s) = /o,l(s)/l,2(s) • • • fd~l,di^) 

PoPi ■ ■ ■ Pd-is'^ PoPi ■ ■ ■ Pd-is'^ 



goA^) det(/rf_i - sPd-i) 
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It is easy to prove that 1 is the unique unit eigenvalue of P, and for i = 0, 2 • • • d — 1, Aj are the 
d non-unit eigenvalues. So 

det(Id_i - sPd_i) = (1 - Aos)(l - Ais) • • • (1 - Arf„is). 

By Lemma 2.1 and the definition of go.dis), 

PoPi ■ --pd-i = s"''/o,d(s)det(/d_i - sPd-i) 

= - Aos)(l - Xis) ■■■{!- Ad„is). 

Let s = 1, because fo,d is a generation function, /o,d(l) = 1- Then 

PoPi ■ --pd-i = (1 - Ao)(l - Ai) • • • (1 - Xd-i). 

As a consequence we have 

(l-Ao)(l-Ai)---(l-Ad-i)s'^ 



(l-Aos)(l-Ais)---(l-Arf„is) 

n 



i=0 



(1-A,)s 



1 - XiS 



3 Proof of Theorem 1.2 

Denote the generator Q of the skip-free Markov chain as 



/ -70 

/5i,o 



Q 



-71 



1,1 



^d- 



1,2 



V 



-7d-l "d-1 

/ 



(a!+l)x(c(+l) 



□ 



and for < n < d— 1, Qn denote the sub-matrix of the first n + 1 rows and first n + 1 lines of Q. 
Ti^i^j be the hitting time of state i + j starting from i. The idea of proof is similar as Theorem 
1.1. We just give a briefly description here. 

It is well known that the skip-free chain on the finite state has an simple structure. The 
process start at i, it stay there with an Exponential (7i) time, then jumps to i+1 with probability 



— , to i — k with probability 



{1 <k <i). Let fi^iJ^j{s) be the Laplace transform of Ti^iJ^j, 



Recall that if a random variable ^ ~ Exponential (0), 



+ s 
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By decomposing the trajectory at the first jump, 

T / \ In Oi-n , Tn l^n,n~l T / \ , l^n,n—2 T / \ , 

Jn,n+l(s) = \ \ \ /„_i,„+i(s) H ■ Jn-2,n+l\s) + 

7n + S 7n In + S 7„ In + S 7n 

\ \ JO,n+l(s) 

7n + S 7n 

C^n , /3n,n—l T / \ i Pn,n—2 T ^ \ , i /-^»i,0 T / \ 
H : Jn-l,n+l(sj H ■ /„_2,n+l(sj H \ ■ /o,n+l(s)- 



Define 



7n + S 7n + S ' 7n + S ' 7n + ' 

5o,o(s) = 1, 9i,i+j{s) = ~ — , for 1 < J < d - z. 

fi,i+j[S) 



The following lemma can be proved by use the method similar as Lemma 2.1, we omit the 
details. 

Lemma 3.1. 

5o,n+i(s) = det{sln - Qn), for < n < d - I. 

Then we can calculate the Laplace transform of ro^rf, recall that Aq, . . . , Xd-i are the d non- 
zero eigenvalues of —Q, we can see they are not equal to easily. And we have 

aocti • • • ad-i = AqAi • • • Xd-i- 

So 

'Pdis) = 7o,l(s)7l,2(s) • • • Jd-lA^) 

d~l , 

ao"i ■ ■ ■ Od-i TT Xi 



nr 



det(s/d_i - Qd~i) fJ-Q 1 - AiS 
complete the proof. □ 
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